fix: improve initial video quality by setting x-google-start-bitrate for all video codecs#973
Conversation
|
…for all video codecs - Apply x-google-start-bitrate SDP hint to all video codecs (VP8, VP9, AV1, H264, H265), not just SVC codecs - Use 90% of target bitrate as start bitrate to prevent initial blurriness - Default degradationPreference to MAINTAIN_RESOLUTION for video tracks to prefer frame drops over resolution reduction when bandwidth is constrained This addresses the issue where video starts blurry for several seconds before improving, by telling WebRTC's bandwidth estimator to start at a higher bitrate instead of ramping up from ~300kbps. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
xianshijing-lk
force-pushed
the
sxian/CLT-3068/fix-initial-video-quality-blurriness-by-setting-x-google-start-bitrate
branch
from
206a3e1 to
3dbe09b
Compare
There was a problem hiding this comment.
This appears to be unrelated to the bitrate/SDP change.
There was a problem hiding this comment.
Thanks, yes, it should removed, it got checked in by mistake
adrian-niculescu
left a comment
adrian-niculescu
left a comment
There was a problem hiding this comment.
The start-bitrate consolidation is a reasonable idea, but a few things need addressing before this lands. Inline notes cover the simulcast bitrate regression, the global degradation-preference default, and the stray patch file.
One more that can't be anchored to a changed line: SdpMungingTest.ensureCodecBitratesTest (livekit-android-test/src/test/java/io/livekit/android/room/SdpMungingTest.kt) still asserts x-google-start-bitrate=700000 for a 1 Mbps target, but the multiplier is now 0.9, so the munge emits 900000. That test fails as-is and needs updating.
| // Handle trackBitrates - apply start bitrate for all video codecs to prevent initial blurriness | ||
| if (encodings.isNotEmpty()) { | ||
| if (finalOptions is VideoTrackPublishOptions && isSVCCodec(finalOptions.videoCodec) && encodings.firstOrNull()?.maxBitrateBps != null) { | ||
| if (finalOptions is VideoTrackPublishOptions && isVideoCodec(finalOptions.videoCodec) && encodings.firstOrNull()?.maxBitrateBps != null) { |
There was a problem hiding this comment.
Switching this gate from isSVCCodec to isVideoCodec pulls simulcast codecs (VP8 by default, plus H264) into this path, but maxBitrate is still taken from encodings.first(). For simulcast, computeVideoEncodings adds encodings smallest-to-largest (the sortedByDescending { calculateScaleDown } ordering), so encodings.first() is the lowest layer. A default 16:9 publish starts at H180 = 160 kbps. registerTrackBitrateInfo feeds that into ensureCodecBitrates, which writes a codec-level x-google-start-bitrate/x-google-max-bitrate from the low-layer value, capping the full-resolution layer far below its target. SVC kept a single full-bitrate encoding, which is why this was SVC-only before. For simulcast you'd need the top layer's bitrate (or to skip the codec-level cap).
There was a problem hiding this comment.
I ran a series of experiments comparing MAINTAIN_FRAMERATE and MAINTAIN_RESOLUTION under constrained uplink conditions using custom Network Link Conditioner profiles.
Network Profiles:
Poor 3G Uplink
Uplink: 300 Kbps
Packet loss: 3%
Downlink: Unbounded
Poor LTE Uplink
Uplink: 3 Mbps
Packet loss: 1%
Downlink: Unbounded
Results
Poor 3G (540p capture)
Both degradation preferences were tested at a 540p capture resolution.
MAINTAIN_FRAMERATE
The first ~10s of video is blurry, and it improves its resolution after that.
MAINTAIN_RESOLUTION
Preserves image clarity by holding a higher resolution.
Don't see much difference on the smoothness of the videos.
Poor LTE (720p capture)
Both degradation preferences were tested at a 720p capture resolution.
The same observation above.
Poor 3G (1080p capture)
An additional experiment was run using a 1080p capture resolution over the 300 Kbps uplink profile.
I start to notice differences of framerate and resolution here. In MaintainFrame mode, the video stays at lower resolution and it does look smoother;
While in MaintainResolution mode, the video stays at high resolution, but less smooth.
I don't see serious choppiness from either modes though.
Poor 3G (both uplink and downlink)
In this experiment, both uplink and downlink were bandwidth constrained, which produced noticeable choppiness regardless of the degradation preference. With the downlink no longer being the bottleneck, the differences between the two modes became much clearer.
Overall, the severe choppiness observed previously appears to have been caused by the network being constrained in both directions rather than by the degradation preference itself.
Personal Impression
When bandwidth is insufficient, the trade off between image quality and motion smoothness is largely subjective. Personally, I slightly prefer maintaining resolution, as I find occasional choppiness less distracting than a combination of choppiness and significant blurriness.
We are still having some discussions internally to figure out the best default option here, and appreciate any input / opinions here.
| override val backupCodec: BackupVideoCodec? = null, | ||
| override val degradationPreference: RtpParameters.DegradationPreference? = null, | ||
| // Default to MAINTAIN_RESOLUTION to prevent initial video blurriness | ||
| override val degradationPreference: RtpParameters.DegradationPreference? = RtpParameters.DegradationPreference.MAINTAIN_RESOLUTION, |
There was a problem hiding this comment.
This flips the default for every video publisher, not just the slow-ramp case. MAINTAIN_RESOLUTION holds resolution by dropping framerate under congestion, which suits screen share but turns camera video into low/stuttering framerate on constrained networks; WebRTC's default for camera content is maintain-framerate. The same change is in VideoTrackPublishOptions below, and the KDoc on the abstract degradationPreference still reads "null value indicates default value (maintain framerate)", which now contradicts the default. Consider scoping MAINTAIN_RESOLUTION to screen share rather than making it the global default, and updating the KDoc.
| @@ -0,0 +1,591 @@ | |||
| diff --git a/gradle/libs.versions.toml b/gradle/libs.versions.toml | |||
There was a problem hiding this comment.
This 591-line patch file looks accidentally committed. It's a separate SDP-parser refactor (JainSDP MediaDescription to a hand-rolled SdpMediaSection), unrelated to the bitrate change, and shouldn't be in the tree. It also trips git diff --check on trailing whitespace. Please drop it before merge.
- Revert isVideoCodec back to isSVCCodec for bitrate registration - For simulcast, encodings are ordered smallest-to-largest, so encodings.first() returns the lowest layer's bitrate (e.g., 160kbps for H180), which would incorrectly cap all layers at that low value - SVC codecs (VP9, AV1) have a single encoding with the full bitrate, so this logic is safe for them - Remove unused isVideoCodec function - Remove accidentally committed munging.patch file Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Thanks for pointing out, I am discussing with the team on the proper fix here.
|
2 participants
Summary
x-google-start-bitrateSDP hint to all video codecs (VP8, VP9, AV1, H264, H265), not just SVC codecsdegradationPreferencetoMAINTAIN_RESOLUTIONfor video tracksProblem
Video starts blurry for 5-15 seconds before improving. This is caused by WebRTC's bandwidth estimator starting at ~300kbps and slowly ramping up to the target bitrate.
Solution
Test plan
🤖 Generated with Claude Code